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TITLE OF THE INVENTION 
Imaging Apparatus 

BACKGROUND OF THE INVENTION 
Field of the Invention 

This invention relates to an imaging apparatus for perfonning signal processing 
depending on the operational modes. 
Description of the Related Art 

A digital still camera retrieves image data obtained by a CCD image sensor mto 
a DRAM or a flash memory and subsequently transfers the image data to a so-called 
personal computer or the like. A major proportion of this type of the digital still 
camera has hitherto been of the type coping with the video graphics array (VGA) 
system. 

Referring for example to Fig. I, this digital still camera 200 includes a CCD 
image sensor 201 for generating image signals, an input processing/image processing 
circuit 202, a memory controller 203 for reading and writing image data, an output 
processing circuit 204 for conversion to an output image of a pre-set system, a finder 
205 for displaying the state of an object at the tune of image shooting, a recording unit 
207 for recording compressed image data over a CPU bus 206 and a 
compression/expansion circuit 208 for compressing/expanding image data. The digital 
still camera 200 also includes a memory 209, formed by, for example, a DRAM, and 
a CPU 2 10 for controlling the overall device. 
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Before startmg the image shooting of an object, the user has to confinn an 
object image displayed on the finder 205. This state is termed a fmder mode. At this 
time, the CCD image sensor 201 sends image signals obtained on photoelectric 
conversion to the input processmg/image processing circuit 202. The input 
processing/unage processmg circuit 202 perfonns the correlated dual sampling 
processing on the unage signals to digitize the rniage signals. The input 
processing/image processing circuit 202 then perfonns pre-set signal processing, such 
as gamma coirection, knee processing or camera processing and routes the processed 
Ullage signals to the memory controller 203. The memory controller 203 then is 
responsive to the control by the CPU 210 to send the image data from the mput 
processmg/hnage processing circuit 202 to the output processing circuit 204. The 
outputprocessing circuit 204 encodes image data in accordance with, for example, the 
National Television System Coimmttee (NTSC) system, and analogizes the encoded 
image data to route the resulting analog data to the fmder 205 . This allows the object 
as an object of image shooting to be mdicated on the fmder 205. 

On the other hand, if the user pushes a shutter button, not shown, to shift to the 
recording mode, the memory controller 203 causes the image data fhmished from the 
input processmg/image processing circmt 202 to be written in the memory 209. The 
CPU 2 10 causes the image data to be read out from the memory 209 and compresses 
the image data from the recording umt 207in the compression/expansion circuit 208 
with compression m accordance with, for example, the Joint photographic Experts 



Group (JPEG) system to record the compressed image data in the recording unit 207. 

If the user performs pre-set processing to shift to the reproducing mode, the 
CPU 210 causes image data to be read out from the recording umt 207 to cause the 
image data to be expanded in JPEG system m the compression/expansion circuit 208 
toroutetheresukingdataviamemoiycontroller203andoutputprocessingcircuit204 

to the finder 205. This causes the as-shot image to be displayed on the finder 205. 

In keeping up with recent outstanding technical progress in the CCD image 
sensor, the resolution of image data is nearly surpassing 1,000,000 pixels. On the 
other hand, it may be feared that the digital still camera of the above-described 
structure cannot sufficiently cope with the image data exceeding 1,000,000 pixels. 

If, for example, the CCD image sensor 201 outputs image signals of high 
resolution in the finder mode, the input processing/image processing circuit 202, 
memory controller 203 or the output processing circuit 204 camiot process image data 
in real-time, such that an miage of the object is displayed on the finder 205 in a frame- 
skipping fashion. This incurs an mconvemence in shooting an hnage of object even 
if the object makes the slightest movement. 

In the recording mode, since in which only multi-pixel image data is recorded 
m the recording umt 207, it is umiecessaiy to perfonn the processing in e.g., the input 
processing/image processing circuit 202. 

That is, in the digital still camera 200, since the pre-set signal processing is 
perfonned on e.g., the mput processing/image processing circuit 202 without regard 
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to the operational mode, the signal processing has not necessarily been efficient 
insofar as the entire apparatus is concerned. 
SUMMARY OF THE INVENTION 

It is therefore an object of the present invention to provide an imaging apparatus 
that is able to perfonn efficient signal processing depending on the operational mode. 

Ob 



In another aspect, the present invention provides a controlling method for a 
signal processing apparatus adapted for transmitting/receiving image data between a 
plurality of signal processing means and storage means for storing image data, the 
signal processing means being adapted for processing the image data in a pre-set 
fashion and for outputting to the control means a request signal for demanding 
fumishment of the iinage data for signal processing or demanding the outputting of the 
processed image data. The controlling method includes selecting, on fumishinent of 
the request signal from the plural signal processing means, one or more of the signal 
processing means which has outputted the request signal, and fimiishing the linage 
data read out from the storage means to the selected signal processing means or 
writing the image data outputted by the selected signal processing means in the storage 



means. 



In still another aspect, the present invention provides an imaging apparatus 
including imaging means, storage means for transiently storing image data from the 



UTiaging means, control means for controlling the writmg/readout of the image data for 
the storage means, a plurality of signal processing means for processing the hnage 
data in a pre-set fashion and for outputting to the control means a request signal for 
demanding fumishment of the image data for signal processing or demanding the 
outputting of the processed image data, and outputting means for outputting image 
data processed by the signal processing means. The control means manages control 
on fhmishment of the request signal to select one or more of the signal processing 
means which has outputted the request signal to fiimish the image data read out from 
the storage means to the selected signal processing means or to write the image data 
outputted by the selected signal processing means in the storage means. 

In yet another aspect, the present invention provides a recording/reproducing 
apparatus including imaging means, input processing means for performing pre-set 
input processing on image data from the imaging means, display processing means for 
displaying image data on display means, first storage means for transiently storing the 
image data from the imaging means, control means for controllmg the wnting/readout 
of the image data for the first storage means, resolution converting means for 
converting the resolution of image data, compression/expansion means for 
compressing/expanding the hnage data and recordmg/reproducing controllmg means 
for causing the compressed image data to be recorded on second storage means and 
for causing the image data recorded on the second storage means to be reproduced. 
The control means selects one or more signal processing means from the input 




processmg means, display processing means, resolution converting means and the 
compression/expansion means. The control means causes the image data read out 
from the first storage means to be sent to the selected signal processing means or 
causes the image data outputted by the selected signal processmg means to be written 

in the first storage means. 

In the signal processing apparatus and the control method therefor, according 
to the present invention, if a request signal is sent from each signal processing means, 
the signal processing means which has outputted the request signal having the utmost 
priority in the priority order is selected. Control is then performed for supplying the 
image data read out from the storage means over the image data bus to the selected 
signal processing means, or wntmg the processed image data of the selected signal 
processing means over the image data bus to the storage means, so that efficiently 
signal processing will be executed in the respective signal processing means. 
The present invention provides a An imaging apparatus comprising: 
imaging means for generating image data based on the imaging hght from an 
object; 

memory means for storing the image data; 

a plurality of signal processmg means for performing pre-set signal processing 

on the image data; 

display means for displaying an image corresponding to said image data; 
a recording medium for recording the image data thereon; and 



control means for perfonning control in a first operational mode for processing 
the i.«age data from said imaging means in a pre-set fashion by the signal processing 
means of said plural signal processing irreans required to perform real-time processing 
to write the image data m said meinory mearrs and for reading out the processed image 
data fi-om said memory means to supply the read-out image data to said display means, 
said control means perfonmng control in a second operational mode for writing the 
image data from said imaging means in said memory means and subsequently reading 
out the written image data to route the read-out .mage data to said plural signal 
processing means to record the Unage data processed by said plural signal processing 
means on said recording medimn. 

In the first mode of the imaging apparatus, the image data from the imaging 
means are decimated and processed in a pre-set fashion by signal processing means 
required to perfonn real-time processing. In the second mode, multi-pixel image data 
are first written in the memory means. The multi-pixel image data are then read out 
therefrom and routed to and processed by the plural signal processing means. 

More specifically, the signal processing means of the plural signal processing 
means which is required to perfonn real-tune processing is caused to perfonn pre-set 
signal processing on the hnage data from the imaging means, in the first operational 
mode of the imaging apparatus, the resulting image data being then written in the 
memory means and the processed image data being then read out from the memoiy 
means and routed to the display means. In the second operational mode, the image 



data from the imagmg means is ^^Titten in the memory means and read out therefrom 
so as to be routed to the respective signal processing means for processing, with the 
processed image data being then recorded on the recordmg medimn. This reahzes 
signal processing most efficient depending on the operafional mode. 
BRIEF DESCRIPTION OF THE DRAWINGS 

Fig./i^ a block diagram for illusfrating the structure of a convenfional digital 

still camera. 

Fig./is a block diagram showing a schemafic structure of a digital still camera 

embodying the^resent invention. 

Fig.3/^a block diagram showing the schematic structure of the digital still 

camera shown ^ Fig.2. 

Fig.44 a block diagram for illustrating flow of image data in a signal processing 

unit of the digjtal still camera shown in Fig.2. 

¥ig.ps a block diagram for illustrating the structure of a simphfied resolution 
conversion cii^uit in an input processing circuit of the signal processing unit. 

Fig/is a block diagram showing the structure of the resolution conversion 

circuit of the signal processing unit. 

Fig.v/s a block diagram showing a specified structure of a horizontal direction 
buffer, a horizontal direction conversion processing circuit, a vertical direction buffer 
and a vertical direction conversion processing circmt of the resolution conversion 
circuit. 




Fig.8^a block diagram showing an alternative structure of the resolution 
conversion circuit. 

Fig./i^a block diagram showing the structure of the vertical direction buffer 
of the resolution conversion circuit. 

Fig. 1 (^lustrates a technique for reading out image data from the image memory 

by the memory controller. 

Fig. l/iUustrates the coordinate position of pixels making up an image. 

Fig. Illustrates another techmque for reading out image data from the image 
memory by the memory controller. 

Fig.l/fs a block diagram showing the structure of the horizontal direction 
buffer of the resolution conversion c constituted by a line buffer. 

Fig.l^llustrates the technique when the memory controller reads out image 

data from the image memory. 

Fig.iy(s a block dy4ram showing the structure of the simphfied resolution 
conversion circuit in twijTSC/PAL encoder of the signal processing unit. 

Figs. ]/6A to id show a timing chart for illustrating the contents of the signal 
processing in the respective circuits in the finder mode. 
DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Referring to the drawings, preferred embodiments of the present invention will 

be explained in detail. 

The present invention is applied to digital still camera I, configured as shown 



for exainple in Fig. 2. 

The digital still camera 1 includes an image generating unit 10 for generating 
image signals, an input signal processor 20 for processmg image data m a pre-set 
fashion, an nnage memoiy 32, comprised of an SDRAM, and a controller 40 for 
controlling the input signal processor 20. 

The image generating unit 10 includes a solid-state imaging device for 
generating image signals, such as a CCD image sensor 11, a sample holding- 
anal og/digital circuit (S/H-A/D circuit 12) for sample-holding and digitizing the image 
signals to output image data, and a timing generator 13 for generating timing signals. 
This timing generator 13 generates horizontal synchronization signals and vertical 
synchronization signals for controlling respective circuits of the image generating unit 
10 based on synchronization signals supplied from the signal processor input . 

The CCD image sensor 11 generates image data corresponding to XGA 
(extended graphic array: 1024x768) pixel data made up of, for example, 800,000 
pixels. The CCD image sensor 1 1 is driven based on the synchronization signals from 
the timing generator 13 to output image signals at a rate of 30 frames per second. 
Meanwhile, the CCD image sensor 1 1 has the frmction of thinning out image signals 
and is able to thin out vertical components of the image signals to 1/2, 1/3, 1/4, - to 
output the resulting thinned-out signals. 

The S/H-A/D circuit 12 is also adapted to perform sample-holding and A/D 
conversion at a pre-set samplmg interval based on the synchronization signals from the 
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timing generator 13 to send the resulting image data to the signal processor 20. 

The signal processor 20 includes a sole LSI (large scale integrated circuit). The 
signal processor 20 includes an input signal processor 21 for input processing and 
camera processing on image data from the image generating unit 10, a memory 
controller 22 for controlling the readout/write of image data for the image memory 32 , 
an NTSC/PAL (phase alternation by line) encoder 23, a D/A converter 24 for 
analogizing image data and outputting the resulting analog signals to outside, and a 
sync generator 26 for generating synchronization signals and supplying the resulting 
synchronization signals to the timing generator 13. 

The signal processor 20 also includes a memory interface 27, as an interface for 
the image memory 32, a resolution conversion circuit 28 for converting the resolution 
of the image data, a JPEG (Joint Photographic Experts Group) encoder/decoder 29, 
for compressing/expanding image data, a JPEG interface 30, as an interface of the 
JPEG encoder/decoder 29, and a host interface 31, as an interface for having data 
transmission/reception with the CPU of the controller 40. 

The input signal processor 2 Iprocesses the image data from the S/H-A/D circuit 
12 with digital clamp, shading correction, aperture correction, gamma correction or 
color processing and routes the resulting processed signals to the memory controller 
22. The input signal processor 2 1 has the fiinction of processing input data to convert 
the input data into Y, Cb and Cr. If the resolution of the image data is larger than that 
of the VGA (Video Graphics Array), the input signal processor 21 is able to perfonn 
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the processing of lowering the resolution. The input signal processor 2 1 also perforais 
the auto-focussing and auto-iris detection to route the data to the controller 40 to effect 
automatic adjustment of the focussing mechanism and the iris mechanism. The input 
signal processor 21 also detects the signal level of the three prime colors making up 
the image data to adjust automatic white balance. 

The memory controller 22 also performs control to cause image data supplied 
from the input signal processor 2 1 or other cfrcuitry to be written in the image memory 
32 via a memory interface 27 and to read out image data of the image memory 32 via 
the memory interface 27. At this time, the memory controller 22 detects whether or 
not there is any defective pixel in the CCD image sensor 1 1 based on the image data 
stored in the image memory 32. 

The memory controller 22 routes the image data read out from the image 
memory 32 to, for example, the NTSC/PAL encoder 23. When fed with the image data 
from the memory controller 22, the NTSC/PAL encoder 23 encodes the image data 
in accordance with the NTSC system or the PAL system to send the encoded data to 
the D/A converter 24. The D/A converter 24 analogizes the image data to output the 
resulting analog signals via output terminal 25. 

The memory controller 22 routes the image data, read out from the memory 
controller 22, to the resolution conversion circuit 28 to cause the image data to be 
converted in resolution, while causing the image data outputted by the resolution 
conversion circuit 28 to be written in the image memory 32. 



12 




The memory controller 22 routes the image data via the JPEG interface 30 to 
the JPEG encoder/decoder 29 to effect compression of the still image, while causing 
the image data expanded by the JPEG encoder/decoder 29 to be written in the image 
memory 32. 

The image memory 32 not only stores the image data as described above but 
also stores OSD data (on-screen-display data) as the so-called character generator data. 
The OSD data is made up of bit map data. The controller 22 controls the readout/write 
of the OSD data. The image data and the OSD data are synthesized by the NTSC/PAL 
encoder 23. 

fl The controller 40 includes a CPU (central processing unit) 4 1 for controlling 

1 the respective circuits of the signal processor 20, a DRAM (dynamic random access 

5 memory) 42, a ROM (read-only memory) 43, having the control program for the CPU 

Z 4 1 stored therein, a flash memoiy interface 44, as an interface for exchanging image 

5 data with a storage device 5 1 , such as a flash memory, and an IrDA interface 45, as an 

% interface of the communication circuit 52 constituted such as by IrLED. 

" For example, the CPU 41 causes hnage data compressed by the JPEG 

encoder/decoder 29 to be written via a flash memory/interface 44 in a storage device 
51, made up of a flash memory, while causing image data to be read out from the 
storage device 5 1 to route the hnage data read out from the JPEG encoder/decoder 29. 
The CPU 41 also causes the image data read out from the storage device 51 to be 
outputted via the IrDA interface 45 and the communication circuit 52 as infrared light 
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to outside. 

The schematic structure of the digital still camera 1 is shown in Fig.3. 
The mput signal processor 2 1 routes the image data from the CCD image sensor 
1 1 via an image data bus 33 to the huage memory 32. The NTSC/PAL encoder 23 
encodes the image data from the image memory 32 in a pre-set fashion to send the 
resulting encoded data to the finder 36. This causes an image of an object to be 
displayed on the finder 36 which is adapted to display the image in association with 
the image data up to the VGA fonnat. 

The memory controller 22 performs data transfer between the image memoiy 
32 and the signal processing circuits connecting to the image data bus 33. The 
resolution conversion circuit 28 performs resolution conversion of the image data from 
the image memory 32 to route the results to the image memory 32. The JPEG 
encoder/decoder 29 compresses the image data from the image memory 32 in 
accordance with the JPEG system to route the compressed image data via CPU bus 34 
to the CPU 41, which then causes the compressed image data to be written in the 
storage device 5 1 . The CPU 4 1 is also able to output the compressed hnage data via 
the CPU bus 34 and the coimnunication circuit 52 to outside. 

Thus, is Fig.3, the respective circuits of the signal processor 20 are 
mterconnected over the image data bus 33. The image data bus 33 is a virtual bus and 
indicates that there is placed a limit to the transfer band for image data exchanged 
between the respective circuits. 



14 



In the signal processor 20, the respective circuits, such as NTSC/PAL encoder 
23 or the resolution conversion circuit 28, send to the memory controller 22 a request 
signal mdicating that unage data are demanded. These cu-cuits also transmit a request 
signal to the memory controller 22 when outputting the image data after the end ofthe 

processing of the image data. 

On reception ofthe request signals from the respective circuits, the memory 
controller 22 selects those circuits having the high priority sequence, and transmits an 
acknowledge signal to the selected circuit. The acknowledge signal indicates that 
hnage data can he routed to a circuit receiving the signal or that image data outputted 
hy a circuit which has received the acknowledge signal is ready to be received. The 
memory controher 22 reads out image data from the unage memory 32 to route the 
read-out image data via image data bus 33 to the circuit corresponding to the 
destination ofthe acknowledge signal. The memory controller 22 receives the image 
data outputted by the circuit which has sent the acknowledge signal to write the image 

data in the image memory 32. 

On reception ofthe request signals from plural circuits, the memory controller 
22 is able to select preferentially the circuit which has to perfonn the processing in 
real-time. For example, if an image of an object is to be displayed on the finder 36, 
the memory controller 22 preferentially selects the input signal processor 2 1 and the 
NTSC/P AL encoder 23. It is also possible for the memory controller 22 to decipher 
the bus occupation ratio ofthe image data on the hnage data bus 33 to detennine the 
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priority sequence of the respective circuits depending on the occupation ratio. 

If unage data can be routed to the respective circuits within the transfer band 
Umitation of the unage data bus 33, it is possible for the memory controller 22 to 
perfonn control to send the acknowledge signal to the respective circuits thne- 
divisionally to permit the respective circuits to perform pre-set processing. This 
enables the memory controller 22 to have access in real-time to data in the respective 
circuits to cause the image data from the respective circuits to be written in the image 
memory 32 or to cause the image data in the image memoiy 32 to be read out and sent 

to the respective circuits. 

If, when the memory controller 22 has access to external circuitry, not shown, 
over the huage data bus 33, the external chcuitry can send the above-mentioned 
request signal or receive the transmitted acknowledge signal, the memory controller 
22 can have access simultaneously and time-divisionally to the respective circuits 
within the signal processor 20 within the transfer band limitation range of the image 
data bus 33. That is, if within the range of the band of the image data bus 33, the 
memory controller 22 can have simuhaneous access to the circuits in the signal 
processor 20 or to the external circuits within the signal processor 20 time-divisionally 
without regard to the number of the circuits within the signal processor 20 or the 
external circuit. 

As mentioned above, the memory controller 22 perfomis arbitration of the 
image data bus 33, write/readout control of image data between the image memory 32 
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and the respective circuits and data transfer to the CPU bus 34. 

The specified flow of image data in the signal processor 20 is explamed with 

reference to Fig. 4. 

The input signal processor 2 1 includes a CCD interface 2 1 a for perfonnmg pre- 
set signal processing on the image data from the image generating unit 10, a detection 
circuit 2 lb for processing the CCD interface 2 1 a, and a camera digital signal processor 
21c (camera DSP 21c) for doing conversion processing of the image data. 

The CCD interface 21a performs the processing, such as the digital clamp, 
white balance adjustment or gamma correction, on the image data made up of R, G and 
B from the S/H-A/D circuit 12 shown information Fig.2, or decimates the components 
in the honzontal direction of image data in case of necessity. After such processing, 
the CCD interface 21a routes image data to the camera DSP 21c or to the memory 
controller 22 via the image data bus 33. 

From the image data of the CCD interface 21a, the detection circuit 21b 
perfonns detection for auto-focussing, auto-iris or white balance adjustment. 

The camera DSP 21c converts the image data of R, G and B from the CCD 
interface 21a into image data made up of luminance signal Y and chrominance signals 
Cb, Cr. The camera DSP 21c also has a simphfied resolution conversion circuit 
2 1 which not only perfonns the above processing but also converts the resolution of the 
image data in a simplified fashion. 

The simplified resolution conversion circuit 2 Id operates for converting the 



17 



resolution of the image data to lower values if the resolution of the huage data 
generated by the CCD image sensor 1 1 is larger than, for example, the VGA format. 
Specifically, the simphfied resolution conversion circuit 2 1 d includes a B - Y/R 

- Y separation circuit 61, for separatmg chrommance signals, a horizontal direction 
linear interpolation ckcuit 62 for interpolation in the horizontal direction, a B - Y/R 

- Y synthesis circuit 63 for synthesizing the chrominance signals, a IH delay circuit 
64 for delaying the respective signals by a horizontal scanning period (IH period), and 
a vertical direction linear interpolation circuit 65 . 

The B - Y/R - Y separation circuit 6 1 separates the chrominance signals B - 
Y and R - Y, as chroma signals Cb, Cr, from the image data from the camera DSP 2 Ic 
, to route the separated chroma signals to the horizontal direction linear interpolation 
circuit 62. The horizontal dfrection linear interpolation circuit 62 interpolates the 
luminance signals Y and the chrommance signals B - Y, R - Y in the horizontal 
direction to lower the luminance m the horizontal direction to route the interpolated 
luminance signals Y and the chrominance signals B - Y, R - Y to the B - Y/R - Y 

synthesis circuit 63. 

The B - Y/R - Y synthesis circuit 63 synthesizes the chrominance signals B - 
Y, R - Y, to route the luminance signals Y from the horizontal direction linear 
interpolation circuit 62 and the synthesized chrominance signals B - Y, R - Y to the 
IH delay circuit 64 and to the vertical direction linear interpolation circuit 65 . The IH 
delay circuit 64 delays the luminance signals Y and the chrominance signals by IH to 



route the delayed signals to the vertical direction linear interpolation circuit 65. The 
vertical direction hnear interpolation circuit 65 performs linear interpolation 
processing in the vertical direction, based on the luminance signals Y and the 
chrominance B - Y, R - Y from the B - Y/R - Y synthesis circuit 63 and the IH delay 
circuit 64, to output image data made up of luminance signals Y' and chroimnance 
signals (B - Y)', (R - Y)' lowered in resolution in both the horizontal and vertical 
directions. 

The resolution conversion circuit 28 perfonns resolution conversion processing 
of converting [pxq] image data into [mxn] image data. The resolution conversion 
circuit 28 perfonns processing for suppressing the resolution to a pre-set value if the 
rniage data produced in the CCD image sensor 1 1 are of high resolution. It is however 
possible to process the image data of low resolution into data of high resolution. 

Referring to Fig.6, the resolution conversion circuit 28 includes an input buffer 
71 for storing image data, inputted from the image data bus 33, a horizontal direction 
buffer 72, for buffering the image data from the an input buffer 71 in the horizontal 
direction, a horizontal direction transform processing circmt 73 for converting the 
resolution of the image data from the horizontal direction buffer 72 in the horizontal 
direction, a vertical direction buffer 74 for buffering the image data from the 
horizontal direction transform processing circuit 73 in the vertical direction, a vertical 
direction transfonn processing circuit 75 for convertkg the resolution of the hnage 
data in the vertical direction, and an output buffer 76 for buffering at the time of 
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outputting. 

When ready for converting the resolution of the hnage data, the resolution 
conversion circuit 28 outputs a read request signal requesting the memoiy controller 
22 to read out hnage data from the image memory 32, while outputting a write request 
signal requesting the memoiy controller 22 to write the image data in the image 
memory 32 after the conversion processing of the image data. The resolution 
conversion circuit 28 also receives an acknowledge signal indicatmg that the memory 
controller 22 has responded to the request signal. 

Referring to Fig.7, the horizontal direction buffer 72 is made up of a first delay 
circuit 8 1 , a second delay circuit 82 and a third delay circuit 83 each for producing the 
delay of one pixel. Thus, the first delay circuit 81 outputs image data delayed by one 
pixel, while the second and thh-d delay circuits 81, 82 output image data delayed by 
two pixels and image data delayed by three pixels, respectively. 

Referring to Fig.7, the horizontal direction transfonn processing circuit 73 
includes first to fourth multiphers 84, 85, 86, 87, and fnst to third adders 88, 89, 90. 
A circuit for nonnalizing data is incidentally annexed at back of the adder 90. 

The fu-st multiplier 84 multiplies the image data supplied from the an input 
buffer 71 with a pre-set coefficient to route the resulting data to the adder 88. The 
second multiplier 85 multiplies the image data supphed from the fnst delay circuit 81 
with a pre-set coefficient to route the resulting data to the adder 88. The third 
multiplier 86 multiphes the image data supplied from the second delay circuit 82 with 
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a pre-set coefficient to route the resuhing data to the adder 89. The fourth muhipher 
87 muhiphes the nnage data suppUed from the third delay circuit 83 with a pre-set 
coefficient to route the resuhing data to the adder 90. The fn-st adder 88 synthesizes 
the image data to send the resulting data to the second adder 89. The second adder 89 
synthesizes the image data to send the resulting data to the third adder 90. The third 
adder 90 synthesizes the respective image data to send the resuhing data as image data 
converted m resolution in the horizontal direction to the vertical direction buffer 74. 

Thus, the horizontal direction transfomi processing circuit 73 weights plural 
image data each having one pixel delay in a pre-set fashion with pre-set weights and 
synthesizes the weighted image data to interpolate or decimate the pixels in the 
horizontal direction to convert the resolution in the horizontal direction. 

The vertical direction buffer 74 is constituted by a serial connection of first to 
third buffers 9 1 , 92, 93, each adapted to produce a one-line delay. Thus, the fnst bufer 
memory 91 outputs image data delayed by one line, while the second and third buffer 
memories 92, 93 output the image data delayed by two and three lines, respectively. 

Referring to Fig.7, the vertical direction transfomi processing circuit 75 
includes fifth to eighth multipliers 94 to 97 and fourth to sixth adders 98 to 100. The 
vertical direction transfonn processing circuit 75 occasionally includes a circuit for 
normalizing data on the downstreain side of the adder 90. 

The fifth multiplier 94 multiplies the image data supplied from the horizontal 
direction conversion circuit 73 with a pre-set coefficient to route the resulting data to 
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the fourth adder 98. The sixth muhiplier 95 muhiphes the image data supphed from 
the first line memory 91 with a pre-set coefficient to route the resulting data to the 
fourth adder 98. The seventh multiplier 96 multiphes the image data supplied from the 
second line memory 92 with a pre-set coefficient to route the resulting data to the fifth 
adder 99. The eighth multiplier 97 multiplies the hnage data supplied from the third 
line memory 93 with a pre-set coefficient to route the resuhing data to the sixth adder 
100. The fourth adder 98 synthesizes the image data to send the resulting data to the 
fifth adder 99. The fifth adder 99 synthesizes the image data to send the resuhing data 
to the sixth adder 100. The sixth adder 100 synthesizes the respective hnage data to 
output the resulting data as image data converted in resolution in the horizontal 
direction. 

Thus, the vertical direction transforai processing circuit 7 5 weights plural image 
data each having one line delay in a pre-set fashion with pre-set weights and 
synthesizes the weighted image data to mterpolate or decimate the pixels in the 
horizontal dhection to convert the resolution in the vertical direction. 

In Fig.7, the resolution conversion circuit 28 fu-st performs resolution 
conversion in the horizontal direction followed by resolution conversion in the vertical 
direction. It is however possible for the resolution conversion circuit 28 to perfonii 
resolution conversion in the vertical direction followed by conversion in the horizontal 
direction. That is, the resolution conversion circuit 28 may be configured to supply 
the h-nage data from the input buffer 7 1 to the vertical direction buffer 74 and to effect 
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the processing in the vertical direction buffer 74, vertical direction transform 
processing circuit 75, horizontal direction buffer 72 and in the horizontal direction 
transform processing circuit 73, in this order. 

In the above-described embodhnent, the first to third buffer memories 91 to 93 
in the vertical direction buffer 74 are configured to store one-line (IH) image data. 
Alternatively, the first to third buffer memories 9 1 to 93 may be configured for storing 
image data lesser than one line, as shown in Fig.9. It is then necessary for the memory 
controller 22 to read out the image data stored in the image memory 32 every N pixels, 
as shown in Fig. 10. 

Specifically, the memoiy controller 22 reads out pixel data corresponding to 
a viewing screen stored m the image memory 32 every N pixels on the line basis in the 
vertical direction. Referring to Fig. 1 1 , each viewing screen is made up of pxq pixels, 
with the coordinate of the upper left pixel being (1,1), that of the upper right pixel 
being (p, 1), that of the lower left pixel being (l,q) and with the lower right pixel being 

(p,q)- 

Referring to Fig. 12, the memory controller 22 causes the image data of N pixels 
to be read out on the line basis in the horizontal direction in the sequence of the rows 
1, 2, q. This causes the memory controller 22 to read out image data 
corresponding to N pixels from the left end, or Nxq pixels, that is pixel data in an area 
defined by (1,1), (l,q), (N,q) and (N,l). This hnage data is referred to below as unage 
data set (1). 
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Tlie memory controller 22 then reads out image data in a range defined by (N- 
1) q)^ (2N-2, q), (2N-2, 1), referred to below as the image data set (2). If the 
memory controller 22 reads out the image data set (1) and the image data set (2), it is 
tantamount to reading out the image data ofthe (N-l)st column and the Nth column 
twice. 

The reason is that, since the vertical direction transfonn processing circuit 75 
perfonns interpolation beginning fi-om the surrounding pixel, the pixels stored in the 
beginning end and the trailing end ofthe first to thkd buffer memories 9 1 to 93 are not 
the object of processing. For example, if the hnage data set (1) is read out, the pixel 
(N, 1) is not the object of the interpolation processing in the vertical direction. 
However, this pixel (N, 1) is read out when the pixel data set (2) is read out, and 
becomes the object of interpolation processing. 

In similar manner, the memory controller 22 reads out image data of N pixels 
in the horizontal direction every line so that image data of the last two columns ofthe 
directly previous image data set will be included. This routes the image data set to the 
resolution conversion circuit 28. 

The vertical direction buffer 74 is fed with image data, in an amount 
correspondmg to the capacity ofthe first to third buffers 91 to 93, on the line basis. 
Thus, image data offset one line is stored in each f the first to third buffer memories 
91 to 93. The vertical direction transform processing circuit 75 is able to perfonn the 
resolution conversion processing in the vertical direction based on the image data from 
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the first to third buffers 91 to 93 of the vertical direction buffer 74. 

With the memory controller 22, the memory controller 22 can cause the 
resolution conversion circuit 28 to execute the resolution conversion in the vertical 
direction, by readout m meeting with the capacity of the buffer memory, even if the 
capacity of the buffer memory required for resolution conversion in the vertical 

direction is not up to one hne. 

Although the read-out overlap between the image data sets is two columns, it 

is probable that the overlap exceeds two colunms or there is no overlap. It is noted 

that the present invention is applicable to image signal processing, such as camera 

signal processing, without limitation to resolution conversion. 

Although the foregoing description is directed to the embodiment in which the 

buffer memory is being used for interpolation for the vertical direction, the present 

invention is also applicable to an embodiment in which the buffer memory is being 

used for interpolation for the horizontal direction. 

That is, the resolution conversion circuit 28 may perfonn resolution conversion 
in the horizontal direction using a horizontal direction buffer 72a comprised of a buffer 
memory 72a having a capacity of N pixels, as shown in Fig. 13. The memory 
controller 22 can read out uuage data of N pixels on the colunm basis in the sequence 
of the rows 1, 2, p in the vertical direction, as shown in Fig.l4. Meanwhile, it is 
necessary for the memoiy controller 22 to read out the image data stored at the 
leading and trailing ends ofthe buffer memory twice, as in the above-described vertical 
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interpolation processing, so that these hnage data will be the object of the horizontal 

interpolation processing. 

Thus, the memory controller 22 is able to read out uuage data from the image 
memory 32 so that resolution conversion processing in the horizontal and vertical 
directions will be effected for the first to third buffer memories 91 to 93 each having 
a capacity of N pixels. This enables the circuit scale of the horizontal direction buffer 
72 and the vertical direction buffer 74 to be reduced to lower the production cost. 

The NTSC/PAL encoder 23, executing the encoding as described above, also 
has a simphfied resolution conversion circuit 23a for increasing the resolution of the 
unage data, if need be, before proceeding to encoding. 

The sii-nplified resolution conversion circuit 23a perfonns resolution conversion 
for matching to the display standard of the finder 36 if the image data on the image 
memory 32 is lower than the resolution required for display. 

Referring to Fig. 1 5, the simplified resolution conversion circuit 23a mcludes a 
line memory 101 for storing image data from the image data bus 33, a vertical 
direction linear interpolation circuit (V-direction linear interpolation circuit 102) for 
interpolating hnage data in the vertical direction, and a horizontal direction 
interpolation circuit 103. 

The line memory 101 stores image data from an mput terminal in in an amount 
corresponding to one line to send the image data to the V-direction linear interpolation 
circuit 102 in the order it is stored. The V-direction Imear interpolation circuit 102 
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ights the image data from the input temiinal in and the image data from the V- 



direction linear interpolation circuit 102 with a pre-set weighting to perfonn linear 
interpolation in the vertical direction. The horizontal direction interpolation circuit 
103 interpolates Y with an order-seven filter, while interpolating Cb and Cr with an 
order-three fiher. This is simply the interpolation for increasing the resolution by a 
factor of two. The horizontal direction interpolation circuit 103 outputs the image data 

at an output terminal out. 

For example, if image data inputted from the input tenninal in is denoted a, 
image data read out from the line memory 101 is b, a coefficient for weighting is g, 
where 0 < g ^ 1, and image data outputted by the V-direction Imear interpolation 
circuit 102 is c, the V-direction linear interpolation circuit 102 effectuates the 
following processing: 

c = g*a+(l-g)*b. 

The image data outputted by the output tenninal out is encoded by the 
NTSC/PAL encoder 23, as mentioned previously. 

In the signal processing system, the digital still camera 1 is made up of so-called 
two chips, namely s signal processor 20 and a CPU 41. Therefr)re, the respective 
signal processing circuits are each of the chip configuration, so that the substrate 
surface area and frirther the power consumption can be made smaller than if the 
respective signal processing circuits are of separate chip configurations. 

Also, since the signal processor 20 is not of the chip configuration inclusive of 
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the CPU, signal processing can be adaptively effectuated even if the appHcation in 
connection with the CPU 41 is changed. That is, if the signal processor 20 is of the 
chip configuration inclusive of the CPU, it is impossible to reconstruct the chip in case 
the application of the CPU is changed. However, the signal processor 20 can perform 
the pre-set signal processing using a CPU of an optimum structure on the application 
basis. 

The digital still camera 1 of the above-described structure has a finder mode for 
confmning the status or the position of an object prior to image shooting, a recording 
mode for shooting the image of the object as confmned, and a reproducing mode for 
confirming the shot state of the object image, and effects the processing depending on 

the prevailing mode. 

In the fmder mode, the user has to observe the state of the object indicated on 
the fmder 36 before thrusting a shutter button, not shown, to shoot the object. In this 
finder mode, the memory controller 22 and other circuits are controlled in the 
following manner. For illustrating the respective modes, reference is had mainly to 
Fig.4 and occasionally to Fig. 16. 

In the fmder mode, the CCD image sensor 1 1 generates image signals, thinned 
out to one-third fi-om the vertical components, and furnishes the digitized unage data 
via the S/H-A/D circuit 12 to the CCD interface 21a. 

The CCD interface 21a performs signal processing in synchronism with clocks 
shown in Fig.l6A. Specifically, the CCD interface 21a decimates the horizontal 
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components of the image data supphed by the image generating unit 10 to one-third 
and corrects the decimated image data for gamma to send the gamma-corrected data 
to the camera DSP 21c. The CCD interface 21a fhmishes the image data converted 
to 340x256 from the 1/3 decimation process to the camera DSP 21c. 

The camera DSP 21c perfonns data conversion processing on the decimated 
image data into YCrCb image data. The camera DSP 21c converts the resolution of 
the image data in the simplified resolution conversion circuit 2 Id (340x256 - 
320x240) for lowering the resolution of the image data to route the converted image 
data via image data bus 33 to the memory controller 22. 

It is noted that the simplified resolution conversion circuit 2 Id lowers the 
resolution in a shnpHfied fashion to an extent necessary for subsequent processing. In 
this manner, if image data generated by the CCD image sensor 1 1 is of high resolution, 
the transfer range taken up by the image data generated by the CCD image sensor 1 1 
can be decreased to evade the stagnancy on the image data bus 33 to maintain the real- 
time characteristics of the finder mode. 

The memory controller 22 writes the image data in the image memory 32, while 
reading out the image data from the image memory 32 as shown in Fig. 1 6D to send the 
read-out image data via the image data bus 33 to the NTSC/PAL encoder 23. 
Simultaneously, the memory controller 22 reads out the OSD data stored in the image 
memoiy 32, as shown in Fig. 16E, to send the OSD data stored in the image memory 
32, as shown in Fig. 16E. Fig. 16 F shows the state of transfer on the image data bus 
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33 which enables the above-described real-time processing. 

The NTSC/PAL encoder 23 perfonns resolution conversion of 320x240 - 
640x240 or 320x240 - 640x288 in the case of the NTSC system or the PAL system, 
respectively, to send the converted image data to the NTSC/PAL encoder 23. The 
NTSC/PAL encoder 23 also converts the image data into data of the NTSC system or 
the P AL system into OSD data which is routed to the finder 36 shown in Fig.3. This 
allows the image of the object and the title infonnation etc to be displayed in-real thne 
on the finder 36. 

Meanwhile, the NTSC/PAL encoder 23 converts the resolution so that data with 
low resolution will be increased in resolution, such that, if 320x200 image data is 
fimiished, it is converted into 640x240 image data and into 640x288 image data for 
the NTSC system and for the PAL system, respectively. 

In the digital still camera 1, the resolution of the image data generated by the 
CCD image sensor 1 1 is lowered in a simplified fashion in the finder mode to reduce 
the data volume, so that the image data will be within the bandwidth limitation of the 
image data bus 33 and so that the resolution will be increased at an output stage to the 
extent that is necessary for display, at a timing shown in Fig. 16F. 

Thus, with the digital still cameral, the image data is held in the bandwidth 
limitation of the image data bus 33 to permit the image of the object to be displayed 
on the finder 36, even if the image data is of high resolution, without the necessity of 
perfonning the time-consuming decimation processing. 
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If the circuitry for preferential processing, nainely the CCD mterface 21a, 
camera DSP 2 Ic or the NTSC/PAL encoder 23, is previously set in the CPU 41, and 
signal processing is carried out time-divisionally in other circuits as in the above 
circuits, the processing of the respective circuits with high priority may be 
preferentially perfonned depending on the data volume of the image data. 

In the event of the large data volume of the image data in the simplified 
resolution conversion circuit 2 Id, data processing may be perfonned at a high 
processing speed, in order to give priority to real-time processing, even though the 
picture quality is degraded to a certain extent, under control by the CPU 41. In this 
manner, high-speed processing can be effected in the finder mode even in case of the 
large data volmne of the image data generated in the image generating unit 10. 

In the case of the digital still camera 1, having an electronic zooming function, 
the CPU 41 can control the respective circuits in the following manner. 

The memory controller 22 causes the image data, supplied via the CCD 
interface 21a and camera DSP 21c, to be written in the hnage memory 32, while 
causing the image data to be read out from the image memory 32 and routed to the 
resolution conversion circuit 28. The resolution conversion circuit 28 formulates 
image data enlarged from a portion of the input image, by an electronic zooming 
function, to output the resulting iinage data to the unage memory 32. This image data 
is read out from the hnage memory 32 and outputted to the finder 36 via the 
NTSC/PAL encoder 23. This generates electronically zoomed image data. 
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Since the finder mode gives utmost priority to the real-time characteristics, 
time-consuming processing is not executed by the respective circuits. However, the 
CPU 41 can be configured to cause the memory controller 22 and other circuits to 
perfon-n various processing operations if within the range allowed by the transfer area 

of the image data bus 33. 

For example, the memory controller 22 may be configured to read out image 
data from the image memory 32, in which is stored the image data furnished from the 
CCD interface 2 1 a, and to fiimish the read-out image data to the NTSC/PAL encoder 
23 over the image data bus 33 and to the JPEG encoder/decoder 29. The fmder 36 
displays the image of the object in rea-time, while the JPEG encoder/decoder 29 
compresses the image data in accordance with the JPEG system. 

The JPEG encoder/decoder 29 compresses/expands the still image, while it 
cannot process high-pixel image in real-time. It is thus possible for the JPEG 
encoder/decoder 29 to decimate a pre-set number of frames of the image data supphed 
from the image data bus 33 (number of frames or fields) by way of compression or to 
slice a portion of the image to lower the resolution by way of compression. This 
enables shooting of a frame-decimated still image continuously or shooting of a low- 
resolution image continuously. 

The user observes the state of the object displayed on the fmder 36 in the 
above-mentioned fmder mode. If the object is decided to be shot, the user pushes a 
shutter button, not shown. 
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If the shutter button is pushed, the digital still camera 1 proceeds to the 
recording mode. In the recording mode, the CPU 4 1 controls the memory controller 



object on a recording device 51. 

The CCD image sensor 1 1 halts the decimation operation in synchronism with 
the thrusting the shutter button to generate image signals of the XGA format to route 
the digitized image data via the S/H-A/D circuit 12 to the CCD interface 21a. 

The CCD interface 21a routes the image data furnished from the S/H-A/D 
circuit 12 not to the camera DSP 2 Ic, but to the memory controller 22 via the hnage 
data bus 33. The memory controller 22 first writes the image data in the image 
memory 32 and subsequently reads out the image data to route the read-out image data 
via the image data bus 33 to the camera DSP 2 Ic. The camera DSP 2 Ic converts the 
image data made up of RGB into image data made up of Y, Cb and Cr. 

The camera DSP 2 Ic is fed with image data once written in the image memory 
32. That is, the camera DSP 21c effects data conversion on the image data from the 
image memory 32 instead of on the image data directly supplied from the CCD 
interface 21a. Thus, it is unnecessary for the camera DSP 21c to perforai high-speed 
data conversion, but it is only sufficient if the camera DSP 21c executes such 
processing when the image data bus 33 is not busy. Stated differently, it is unnecessary 
for the camera DSP 2 1 c to perform the processing in real-time, so that data conversion 
processing can be executed with priority given to the high picture quality rather than 



22 or the respective circuits in the following manner to record the image of the as-shot 
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to the high processing speed and the resuhing converted image data may be routed to 
the memory controller 22 via the image data bus 33. The memory controller 22 
causes the image data to be written in the image memory 32. 

The memory controller 22 causes the image data to be read out from the image 
memory 32 to route the read-out image data to the JPEG encoder/decoder 29. The 
JPEG encoder/decoder 29 compresses the unage data in accordance with the JPEG 
system to write the compressed image data in the recording device 5 1 shown in Fig.3. 

If real-time processing is not unnecessary, as during recording, the CPU 41 
pennits the pre-set processing to be executed after writing the image data transiently 
in the image memory 32 to exploit the transfer band of the image data bus 33 to 
process the high-pixel image. 

The CPU 4 1 records the image data of the XGA format directly in the recording 
device 5 1 in the recording mode. It is however possible for the resolution conversion 
circuit 28 to convert the resolution of the image data before recording the image data 
on the recording device 51. Specifically, it is possible to cause the resolution 
conversion circuit 28 to convert the resolution of the image data read out from the 
image memory 32 via the memory controller 22 in meeting with the VGA ( 1 024 x 768 
- 640x480) to pennit the JPEG encoder/decoder 29 to compress the image data to 
record the compressed data in the recording device 5 1 . 

If desirous to confirm the as-shot image after image shooting, the operator 
thrusts the playback button, not shown, for reproducing the as-shot image. 
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If the reproducing button is thrust, the digital still camera 1 moves to the 
reproducing mode. In the reproducing mode, the CPU 41 controls the respective 
circuits in the following manner to read out the image data of the object. 

That is, on detecting the thrusting the reproducing button, the CPU 4 1 reads out 
the image data from the recording device 5 1 and transiently stores the read-out image 
data in the DRAM 42 before routing the data via CPU bus 34 to the JPEG 
encoder/decoder 29. The JPEG encoder/decoder 29 expands the iiuage data read out 
from the recording device 5 1 in accordance with the JPEG system to produce image 
data of the XGA fonnat to route the resulting image data via the image data bus 33 to 
the memory controller 22. 

The memory controller 22 writes the image data on the image memory 32 and 
reds out the image data from the image memory 32 to send the read-out iiuage data via 
the image data bus 33 to the resolution conversion circuit 28. 

The resolution conversion circuit 28 effects resolution conversion so that the 
image data will be in meeting with the VGA format (1024x768 - 640x480 in the 
NTSC system and 1024x768 - 640x576 in the PAL system) to route the converted 
image data over the image data bus 33 to the memory controller 22. The linage data 
then is read from the image memory 32 and routed via the NTSC/PAL encoder 23 to 
the finder 36. This displays an image corresponding to the image data recorded in the 
recording device 5 1 on the finder 36. 

That is, since the image data recorded in the recording device 51 has high 
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resolution, the CPU 41 first lowers the resolution and subsequently routes the image 

data to the finder 36. 

It is also possible for the CPU 4 1 to set, for each of the finder mode, recording 
mode and the reproducing mode, the order of priority of the circuits to be processed 
in preference and to cause the pertinent circuit to execute the processing in accordance 
with the order of priority on movement to one of the modes. This enables the signal 
processing of image data to be executed efficiently depending on the processing 



In the above-described embodiinent, it is assumed that the data being processed 
is the image data equivalent to XGA. It is to be noted that the present invention is not 
limited to this embodiment and can be apphed to, for example, the processing of iinage 
data comprised of one milhon or more pixels. 



contents in each mode. 
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